Fetch from nvidia Megatron-LM by RaymondLi0 · Pull Request #5 · ElementAI/Megatron-LM

RaymondLi0 · 2022-08-03T20:32:25Z

No description provided.

Signed-off-by: Li Ding <liding@nvidia.com>

Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Signed-off-by: Jingyue Wu <wujingyue@gmail.com>

Co-authored-by: Yuzhong Wang <yuzhongw@computelab-frontend-3.nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Signed-off-by: conver334 <conver334@gmail.com>

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…error messages (#5117)

…5125) Signed-off-by: Chen Cui <chcui@nvidia.com>

…5121) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Signed-off-by: LeSingh1 <sshaurya914@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: mokai <mokai@baidu.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Signed-off-by: Siddhartha Raman Sundara Raman <sraman@nvidia.com>

Signed-off-by: Helen Ngo <helenn@nvidia.com>

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>

Signed-off-by: Jingyue Wu <wujingyue@gmail.com> Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>

…_sync (#4953) Signed-off-by: factnn <166481866+factnn@users.noreply.github.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

Signed-off-by: Zhiyu Li <zhiyul@NVIDIA.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>

Signed-off-by: Skand Hurkat <shurkat@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

Signed-off-by: YangFei1990 <feiw@nvidia.com>

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>

Signed-off-by: Laura Dang <laurad@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…are (GLM5.2) (#5099) Signed-off-by: Hollow Man <hollowman@opensuse.org>

Signed-off-by: Siddharth Singh <sidsingh@nvidia.com>

Signed-off-by: Yongqiang Wang <yongqiang.seagull@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>

Signed-off-by: Jingyue Wu <wujingyue@gmail.com> Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

…5696) Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: guihong-nv <guihongl@nvidia.com>

…ses (#5445) Signed-off-by: asadbekXodjayev <matyoqub18@gmail.com> Co-authored-by: asadbekXodjayev <matyoqub18@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Signed-off-by: Björn Buschkämper <bjoern.buschkaemper@gmail.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Signed-off-by: yezhengmao <yezhengmaolove@gmail.com>

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>

Signed-off-by: Jon Barker <jbarker@nvidia.com>

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare February 18, 2026 09:18

liding-nv and others added 29 commits June 1, 2026 17:48

fix mimo optimizer checkpoint metadata restore (#4791)

80cf756

Signed-off-by: Li Ding <liding@nvidia.com>

[mimo] Support bridge fan-out for variable modality tokens (#5062)

de030fc

Signed-off-by: Li Ding <liding@nvidia.com>

Add separate mtp_grad_scale_func for MTP loss scaling (#3459)

b8dcaab

Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

[training migration] Migrate GPT builder (#4741)

5f88438

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Make Mamba conv params direct mixer params (#4899)

35992ba

Signed-off-by: Jingyue Wu <wujingyue@gmail.com>

Update copy-pr-bot.yaml [skip ci]

907ebcd

Update oncall reviewer assignment (#5093)

f5da5ea

Pass explicit process groups to hybrid logging (#4781)

bb192ff

Clean up top-level repository files (#5097)

7d9f259

[main] fix(moe): Fix several bugs for DSA rope and spec. (#3026)

b0eb914

Co-authored-by: Yuzhong Wang <yuzhongw@computelab-frontend-3.nvidia.com>

Move MIMO unit tests into models/mimo (#5063)

c69697d

test: update DeepSeek FSDP2 GB200 memory golden (#5094)

54e1996

Remove DeepEP hardware limit check (#4846)

a1ac2f6

Update transformer-engine dependency to revision 4220403 (#5112)

591605e

ci: make CI resilient to pip/uv network timeouts (#5118)

44f254a

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

ci: treat docker container-removal conflict as flaky (#5120)

d94604e

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Fix GDN DTensor splitting for FSDP checkpointing (#4843)

655a064

Signed-off-by: conver334 <conver334@gmail.com>

Fix MoE aux_loss / z_loss gradient scaling with TP > 1 (#5047)

9ef8a2a

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Update copy-pr-bot.yaml [skip ci]

540ff5d

Update Claude copy workflow to enforce user restrictions and improve …

eac5ca7

…error messages (#5117)

Add advisory process group guidance to Claude reviews (#5111)

a823ea0

build: cap pydantic<2.14 in transformer-engine dependency metadata (#…

3b6bc31

…5125) Signed-off-by: Chen Cui <chcui@nvidia.com>

fix(test): skip scalar-less tensorboard event files in resume checks (#…

20f5290

…5121) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

chore: rotate oncall schedule

64c3fb8

docs: fix contributor guide typo (#4858)

c2a3bec

Signed-off-by: LeSingh1 <sshaurya914@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

ci(unit-tests): split slow unit-test buckets over 15min SLA (#5133)

4d104fd

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Fix DSA indexer loss not averaged across micro-batches (#4070)

ec97c85

Co-authored-by: mokai <mokai@baidu.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Update MINOR version to 19 (#5096)

7a7f7b4

Fix Muon QKV split for gated attention (#4728)

947d6ae

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

sraman-rgb and others added 30 commits July 7, 2026 17:46

Fuse shared expert MLP with grouped GEMM (#5604)

115ce7f

Signed-off-by: Siddhartha Raman Sundara Raman <sraman@nvidia.com>

MoE routing analysis and metrics capture (#5220)

c1560d9

Signed-off-by: Helen Ngo <helenn@nvidia.com>

Add cspades to oncall rotation (#5695)

c797a5e

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>

Add microbatch context helper (#5652)

5dbea46

Signed-off-by: Jingyue Wu <wujingyue@gmail.com> Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

ci: Update test configurations to unify legacy scope names (#5316)

68f4c62

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>

Fix Torch FSDP2 crash: add force_all_reduce kwarg to base finish_grad…

6ee356a

…_sync (#4953) Signed-off-by: factnn <166481866+factnn@users.noreply.github.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Separate mFSDP v2 unit tests (#5640)

72a78d6

Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

chore: rotate oncall schedule

465264b

test(determinism): add determinism tests (#5041)

3ab71ee

Signed-off-by: Zhiyu Li <zhiyul@NVIDIA.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

deprecate common strategy (#5160)

40b1fd3

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

ci: revert unify legacy scope names (#5316) (#5709)

509efe6

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Normalize CRLF in Claude fix commands (#5712)

65d6c23

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>

Remove some barriers in save_checkpoint_and_time (#5557)

2a21b8e

Signed-off-by: Skand Hurkat <shurkat@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Add FSDP NVTX annotations (#5704)

91fcdfe

Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

NCCL EP support (#5129)

2d7060f

Signed-off-by: YangFei1990 <feiw@nvidia.com>

Update base image to nvcr.io/nvidia/pytorch:26.06-py3 (#5632)

64bee49

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>

Refactor RL rollout pipeline (#5491)

027fa4a

Signed-off-by: Laura Dang <laurad@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

[2/2] Wiring cuDNN fused DSA kernels support with THD, CP and IndexSh…

d1cab16

…are (GLM5.2) (#5099) Signed-off-by: Hollow Man <hollowman@opensuse.org>

Triton kernels - avoid recompilation and autotuning in prod (#5608)

f29c747

Signed-off-by: Siddharth Singh <sidsingh@nvidia.com>

Add NeMo Transformer audio encoder model (#5565)

cf2f07d

Signed-off-by: Yongqiang Wang <yongqiang.seagull@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

remove deprecated modules from core/dist_checkpointing (#5134)

a2496aa

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

fix(fsdp): import os in safe_get_rank fallback (#4959)

328a77f

Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>

Add forward all-gather overlap (#5513)

ce8865c

Signed-off-by: Jingyue Wu <wujingyue@gmail.com> Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>

Fix seq_load_balancing loss with inter-document masking and MBS > 1 (#…

e86c262

…5696) Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Avoid X11 master port default (#5299)

779c5b7

Signed-off-by: guihong-nv <guihongl@nvidia.com>

Fix infinite recursion in abstract tokenizer special-id property alia…

1aa880d

…ses (#5445) Signed-off-by: asadbekXodjayev <matyoqub18@gmail.com> Co-authored-by: asadbekXodjayev <matyoqub18@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Set Bert TE spec q/k_layernorm to None (#5687)

2c579b4

Signed-off-by: Björn Buschkämper <bjoern.buschkaemper@gmail.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Set is_first_microbatch when quant_recipe is configured (#5642)

5389d14

Signed-off-by: yezhengmao <yezhengmaolove@gmail.com>

Assign BERT CODEOWNERS to GPT team (#5746)

75a2132

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>

Remove use of exec_module (#5744)

48a887f

Signed-off-by: Jon Barker <jbarker@nvidia.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fetch from nvidia Megatron-LM#5

Fetch from nvidia Megatron-LM#5
RaymondLi0 wants to merge 7651 commits into
ElementAI:load-iterfrom
NVIDIA:main

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

Uh oh!

Conversation

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants